On Job Training: Automated Interpersonal Behavior Assessment & Real-Time Feedback

In the service industry, customers often form first impressions about the organization based on the behavior, perceived personality, and other attributes of the front-line service employees they interact with. Literature has has demonstrated that as a major component of interpersonal communication, nonverbal behavior (NVB) contributes towards shaping the outcome of these interactions and thus is key to determine customer satisfaction and perceived service quality. In this doctoral thesis, we aim to develop a computational framework for hospitality students with the goal of providing feedback for improving the impressions that other people make about them. Towards this, we collected a dataset of 169 laboratory sessions consisting of two role-plays, job interviews and reception desk scenarios, a total of 338 interactions. We present our approaches, results, works in progress, and planned future directions on these problems.